System and method to use text-to-speech to prompt whether text-to-speech output should be added during installation of a program on a computer system normally controlled through a user interactive display

ABSTRACT

An auditory user interactive interface to an application program being installed in the computer controlled system. A routine in an object, in an application program being installed in the computer controlled system for providing an auditory user interface to the program in combination with auditory means for offering the user of the computer controlled system the auditory user interface during installation of said application program, and responsive to the selection of the auditory interface provides the auditory user interface during said installation of the application program. The functional object in the application program being installed includes the routine for providing the auditory interface preferably implemented in Text to Speech (TTS) functions together with an implementation for determining whether the computer controlled system receiving the application program includes a TTS engine compatible with the TTS functions in the application program and a routine for installing a TTS engine compatible with the routine when the computer controlled system does not include the TTS engine.

TECHNICAL FIELD

The present invention relates to user-interactive computer systems,normally accessed through interactive displays and particularly tosystems and methods that are user friendly and provide computer userswith an auditory interface that is easy to use and does not impede thenormal display interface system.

BACKGROUND OF RELATED ART

The past generation has been marked by a technological revolution drivenby the convergence of the data processing industry with the consumerelectronics industry. This advance has been even further accelerated bythe extensive consumer and business involvement in the Internet over thepast several years. As a result of these changes, it seems as ifvirtually all aspects of human endeavor in the industrialized worldrequire human-computer interfaces. These changes have made computerdirected activities accessible to a substantial portion of theindustrial world's population, which, up to a few years ago, wascomputer-illiterate, or, at best, computer indifferent.

However, even more significant than these advances in opening newhorizons to the general population has been the ability of the computerand the Internet to rescue people with even very severe physicalimpairments from lives of limited menial or no productivity. With thecomputer, a person with even slight dexterity or vision may, withsufficient effort, be capable of becoming as fully productive from hisdesktop as a person with full dexterity or vision.

This is possible because the computer may be tailored to the uniqueabilities of each individual. To this end, the computer industry iscontinuously seeking new implementations to bring more and more impairedindividuals into full productivity in the workforce.

With an aging population, there is an increasing need to make computertechnology readily accessible to visually impaired users. In the past,it was originally necessary to set up special computer operating systems“for the blind”. In such an environment, it was necessary for thevisually impaired to arrange for such a set up and then to order from alimited number of special application programs “for the blind”. Becauseof the limited marketing potential for such special applicationprograms, software providers have obviously kept the visually impairedusers out of the mainstream of application programs.

However, as object oriented programming technology developed, it becameeasier to include in more application programs, an object providing aselectable auditory interface for the visually impaired. Suchimplementations still had their limitations. For instance, the computersystem into which the application program was being installed had tohave an auditory engine that was compatible with the routines in theapplication program object. In addition, the visually impaired userstill needed help in the installation of the application program throughthe provision of a starting auditory interface during installation.Currently, this is being provided by a prerecorded auditory step by steppresentation. Using prerecorded auditory presentation limits theflexibility and presentation of options during installation.

SUMMARY OF THE PRESENT INVENTION

The present invention addresses both the problems of: 1) incompatibilityof the auditory interface objects and routines in the applicationprograms being installed with auditory engines in the receiving computersystem; and 2) providing an initial dynamic rather than a prerecordedauditory interface for and during the installation of the applicationprogram.

Accordingly, the present invention is directed to a computer controlledsystem normally interfaced through a user interactive display, andprovides an auditory user interactive interface to an applicationprogram being installed in the computer controlled system. The system ofthe invention comprises a routine, preferably as an object, in anapplication program being installed in the computer controlled systemfor providing a dynamic auditory user interface to the program basedupon Text to Speech (TTS) protocols in combination with TTS means foroffering to the user of the computer controlled system the auditory TTSuser interface during installation of said application program, andmeans responsive to the selection of the TTS interface for installation,for providing the auditory user interface during the installation of theapplication program.

The functional object in the application program being installedincludes the routine for providing the auditory interface, preferablyimplemented in TTS functions, together with means for determiningwhether the computer controlled system receiving the application programincludes a TTS engine compatible with said routine. There are means inthe application program responsive to the means for determining whetherthe computer controlled system includes a TTS engine for installing aTTS engine compatible with the routine when the computer controlledsystem does not include the TTS engine. Further, there are means in theapplication program responsive to the means for determining whether thecomputer controlled system includes a TTS engine for modifying analready installed but not compatible TTS engine to a state compatiblewith the routine.

It is to be noted that all of the above steps are carried out initiallyand dynamically using TTS on installation of the application program,but before the user is given any interactive prompt so that the auditoryuser interface is available and on-line for the initial aural userinstallation prompt that is developed by TTS, e.g. “Do you want a spokeninstallation routine?”. If the user answers “Yes”, the auditoryinterface system commences, usually along with the conventional displayprompts. If the user answers “No”, the system continues the displayinterface during installation and takes steps to restore the computersystem to its original operating state, as will be hereinafterdescribed. In other words, the routine embodied in the applicationprogram will automatically set up a compatible TTS engine and initiatethe auditory routine on it at the beginning of installation so that aninitial auditory prompt to select auditory TTS installation instructioncan be made. If the selection is “No”, then any installed auditoryengine or engine modification is uninstalled to revert the computersystem to the original state so that installation of the application maybe made using the conventional display interface for installationprompts and instructions.

As set forth above, the auditory engine used herein uses theconventional TTS techniques involving means for providing textual datarepresentative of a sequence of the normal displayable user interactivedialog panels for the installation and TTS means for translating each ofthis sequence of dialog panels into each of a corresponding sequence ofinteractive auditory instructions and prompt panels.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be better understood and its numerous objectsand advantages will become more apparent to those skilled in the art byreference to the following drawings, in conjunction with theaccompanying specification, in which:

FIG. 1 is a block diagram of a generalized data processing systemincluding a central processing unit that provides the normal computercontrolled interactive display system, as well as the auditoryinstallation interface that may be used in practicing the presentinvention;

FIG. 2 is a diagrammatic view illustrative of the four stages in theinstallation of the application programs of the present inventionthrough the normal display interface contrasted with the interactiveauditory interfaces provided by these application programs;

FIG. 3 is a flowchart of the steps involved in applying the system ofthe present invention to provide, during the installation, an auditoryuser interactive interface to an application program being installed inthe computer controlled system; and

FIG. 4 is a flowchart of an illustrative running of the steps set up inthe program of FIG. 3.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring to FIG. 1, a typical data processing system is shown that mayfunction as the computer controlled display terminal used inimplementing the system of the present invention of providing, duringthe installation of an application program, an auditory user interactiveinterface to an application program being installed in the computercontrolled system. A central processing unit (CPU) 10, such as one ofthe PC microprocessors or workstations, e.g. RISC System/6000™ seriesavailable from International Business Machines Corporation (IBM), orDell PC microprocessors, is provided and interconnected to various othercomponents by system bus 12. An operating system 41 runs on CPU 10,provides control and is used to coordinate the function of the variouscomponents of FIG. 1. Operating system 41 may be one of the commerciallyavailable operating systems, such as IBM's AIX 6000™ operating system orMicrosoft's WindowsXP™ or Windows2000™, as well as UNIX and other IBMAIX operating systems. Application programs 40, controlled by thesystem, are moved into and out of the main memory Random Access Memory(RAM) 14. These programs include the programs of the present inventionfor providing, during the installation of an application program, anauditory TTS user interactive interface to the application program beinginstalled in the computer controlled system. The auditory TTS engine,which will be discussed in greater detail hereinafter, is part of theoperating system 41 when already present in the computer. However, whenthere is no auditory engine installed in the computer or the installedengine needs to be modified so as to be compatible with the auditoryinstallation routine in the application program, then either theauditory engine itself or the modifications to the already installedengine will be included in the application program 40. Read Only Memory(ROM) 16 is connected to CPU 10 via bus 12 and includes the BasicInput/Output System (BIOS) that controls the basic computer functions.RAM 14, I/O adapter 18 and communications adapter 34 are alsointerconnected to system bus 12. I/O adapter 18 may be a Small ComputerSystem Interface (SCSI) adapter that communicates with the disk storagedevice 20. Communications adapter 34 interconnects bus 12 with anoutside Internet or Web network. I/O devices are also connected tosystem bus 12 via user interface adapter 22 and display adapter 36. Aconventional keyboard 24 (with the standard I/O device for the visuallyimpaired), and mouse 26 are all interconnected to bus 12 through userinterface adapter 22. It is through such input devices that the user mayinteractively relate to the programs of this invention. Display adapter36 includes a frame buffer 39 that is a storage device that holds arepresentation of each pixel on the display screen 38. Images may bestored in frame buffer 39 for display on monitor 38 through variouscomponents, such as a digital to analog converter (not shown) for CRTsand the like for digital displays, e.g. LCD displays. The sound or audiosystem providing the interface to users on which the auditory interfacemay be implemented includes any conventional computer sound card 31 withassociated speakers 33. Conventional sound systems that may be used aredescribed in Chapter 5, pp. 155-186 of the text, Personal ComputerSecrets, published 1999, IDG Books Worldwide, Inc., Foster City, Calif.

Now with reference to FIG. 2, there is illustrated the installationstages in the present invention. Since TTS implementations are referredto, some background may be appropriate. Broadly, it is a known processof converting digital text to a speech output. TTS is used extensivelyfor audio weather alerts and telephone messages and prompting. TTS usesa speech synthesis application that is used to create a sound version ofa computer document. TTS is also used by assistive technology programsfor reading of computer display information for the visually impairedperson. There are numerous TTS products on the market.

In Stage 1, application program 55 is to be installed into computer 50,which is normally interfaced through interactive display interface 51.Built into application program 55 is an object 54 that is set up toinitially offer to the user, through speech, via an auditory routine,the alternative of continuing the whole installation of the applicationprogram via a supplemental auditory sequence of prompts and spokencommands. This object even includes a TTS engine 52 to be installed intocomputers that may not have any engine for generating auditoryfunctions. The application program may also include functions that maybe able to modify an auditory TTS engine already installed in thereceiving computer 50 that is incompatible with the installationauditory routine. At this stage, the installation requirements areexamined and a determination made as to whether any auditory TTS enginealready on the computer 50 meets such requirements. These requirementsmay include the ability to process several languages. In one embodiment,the requirement may include compatibility with the locale in effect atthe time of installation. Thus, the application program installationobject requires a TTS engine that can issue instructions in severallanguages. The application program may have the capability of modifyingthe existing TTS engine in the computer so that the TTS engine isupgraded to issue instructions in the requisite languages or in thelanguage that corresponds to the locale in effect at the time ofinstallation.

Then, in Stage 2, the auditory output is rendered functional using theTTS engine, and there is an initial announcement from speaker 53alerting and aurally prompting the user to take appropriate input actionvia the keyboard to initiate auditory instructions 56 in Stage 3, whichare translated by the TTS engine directly from the prompts that wouldappear as dialog panels 57 on screen 51 of computer 50. Where a userselects to have the auditory instructions 56, it is probably mostconvenient to still proceed with the corresponding display instructionsand prompts 57. However, in the case of the user not selecting spokenoutput, then, once the application program is installed, Stage 4, theinstalled auditory TTS engine or the TTS engine modifications areuninstalled 58 in Stage 4 to restore the receiving computer system toits original state.

Now, with reference to FIG. 3, we will describe a process implemented bya program according to the present invention for providing aninteractive auditory interface during the installation of an applicationprogram. The application program to be installed already includes aninstallation routine for optionally providing this auditory interface,step 71. The routine provides for aurally offering to the user theinitial choice of this auditory interface, step 72. There is provided inthe application program, an auditory TTS engine on which the standarddisplayable text instructions are converted into announced speech incase the computer system in the installation has no compatible speechengine, step 73. Also includable in the application program is afunction for modifying a TTS engine previously installed in thereceiving computer system so that the TTS engine is compatible with theauditory TTS function being installed, step 74. Provision is made, step75, in the application program for the installation of the speech engineof step 73 or the modification of step 74 in order that theannouncements of step 72 may be made. Finally, provision is made for theuninstallation of the speech engine of step 73 or the modification ofstep 74 in the case that the interactive user rejects auditoryinstallations, step 76.

In one embodiment, a provision is made for the uninstallation of thespeech engine of step 73 or the modification of step 74 in the case thatthe interactive user declines an offer to accept a license agreementoffered during the installation of the application program.

Now that the basic program has been described and illustrated, therewill be described with respect to FIG. 4, a simple operation showing howthe program could be run. Initially, step 80, the application programincluding an optional TTS engine and/or TTS engine modification functionis input for installation into a computer system. A determination ismade, step 81, as to whether the computer system already has acompatible auditory TTS engine available to it. There are several waysthat such a TTS engine could be available: it may already be installedon the computer system on which the application program is beinginstalled; a compatible TTS engine may be available on another computersystem associated with the computer system of the installation so thatthe TTS engine may be transferred, step 82; or the application programmay have a function capable of modifying an already installed TTS engineso that it is compatible. If both steps 81 and 82 are “No”, then the TTSengine available in the application program is installed in step 83. Ifstep 82 is “Yes”, then the compatible TTS engine available from theassociated computer system is transferred and installed in step 83. Thenor if step 81 is “Yes”, the initial TTS function in the computer systemis functional and there is announced the choice prompt to the user toselect the auditory installation procedure if desired, step 84. Next, ifstep 85 is “Yes”, the user chooses auditory instructions, then theinstallation proceeds with added auditory instructions, step 86. If“No”, the user does not require auditory instructions, and theinstallation can proceed using the standard interactive displayedinstructions, step 93.

As the installation continues, step 87, the user is given opportunities,decision step 88, to cancel the installation (e.g. the user isdissatisfied with the contractual conditions offered during theinstallation and declines to accept the license agreement, or the usercancels the installation for any reason, or the user aborts theinstallation). If “No”, the user chooses to continue, the installationcontinues and a determination is periodically made, step 89, as towhether the installation is completed. If “Yes”, the process is exited,step 90; if “No”, the process continues via branch “A” back to step 87.Now, if in step 88 the user decides “Yes” to cancel the installation orif the user by a “No” decision in step 85 has rejected TTS instructionsand has proceeded with conventional display installation, then viabranch “B” a final determination is made, step 91, as to whether any TTSengine has been installed. If “Yes”, the TTS engine is uninstalled, step92 before exiting. If “No”, the process is directly exited.

One skilled in the art should appreciate that the processes controllingthe present invention are capable of being distributed in the form ofcomputer readable media of a variety of forms.

Although certain preferred embodiments have been shown and described, itwill be understood that many changes and modifications may be madetherein without departing from the scope and intent of the appendedclaims.

1. In a computer controlled system interfaced through a user interactivedisplay, a system providing an auditory user interactive interface to anapplication program being installed in the computer controlled systemcomprising: a routine in an application program being installed in saidcomputer controlled system for providing a Text to Speech (TTS) userinterface to said program; TTS means for offering the user of thecomputer controlled system said TTS user interface during theinstallation of said application program; and means, responsive to theselection of said TTS interface for installation, for providing said TTSuser interface during said installation of said application program. 2.The computer controlled system of claim 1 further including means fordetermining whether said computer controlled system includes a TTSengine compatible with said routine.
 3. The computer controlled systemof claim 2 further including means associated with said applicationprogram, responsive to said means for determining whether said computercontrolled system includes a TTS engine, for installing a TTS enginecompatible with said routine when the computer controlled system doesnot include said TTS engine.
 4. The computer controlled system of claim2 further including: means associated with said application program,responsive to said determining means determining that the computercontrolled system does not include a compatible TTS engine, for seekinganother compatible TTS system from another computer controlled systemoperatively associated with the computer controlled system; and meansfor installing said another compatible TTS system in said computercontrolled system.
 5. The computer controlled system of claim 3 furtherincluding means, responsive to said TTS means for offering the user ofthe computer controlled system said auditory user interface, foruninstalling said installed TTS engine compatible with said routine whensaid user rejects said offered TTS user interface or when said TTS userdeclines an offer to accept a license agreement or when said TTS usercancels said installation of said application program duringinstallation of said application program.
 6. The computer controlledsystem of claim 4 further including means, responsive to said TTS meansfor offering the user of the computer controlled system said auditoryuser interface, for uninstalling said installed TTS engine compatiblewith said routine when said user rejects said offered TTS user interfaceor when said TTS user declines an offer to accept a license agreement orwhen said TTS user cancels said installation of said application programduring installation of said application program.
 7. The computercontrolled system of claim 1 wherein said means, responsive to theselection of said TTS interface for installation, for providing said TTSuser interface during said installation of said application programcomprises: means for providing textual data representative of a sequenceof displayable user interactive dialog panels; and TTS means fortranslating each of said sequence of dialog panels into each of acorresponding sequence of interactive auditory instruction and promptpanels.
 8. In a computer controlled system interfaced through a userinteractive display, a method providing an auditory user interactiveinterface to an application program being installed in the computercontrolled system comprising: including in an application program beinginstalled in said computer controlled system, a method for providing aTTS user interface to said program; TTS offering, to the user of thecomputer controlled system, said TTS user interface during installationof said application program; and providing said TTS user interfaceduring said installation of said application program responsive to theselection of said interface for installation.
 9. The method forproviding said auditory user interface of claim 8 further including thestep of determining whether said computer controlled system includes aTTS engine compatible with said method.
 10. The method for providingsaid auditory user interface of claim 9 further including the stepassociated with said application program, responsive to saiddetermination of whether said computer controlled system includes a TTSengine, for installing a TTS engine compatible with said method when thecomputer controlled system does not include said TTS engine.
 11. Themethod for providing said auditory user interface of claim 9 furtherincluding the steps, responsive to a determination that the system doesnot include a TTS engine, of: seeking another compatible TTS system fromanother computer controlled system operatively associated with thecomputer controlled system; and for installing said another compatibleTTS system in said computer controlled system.
 12. The method forproviding said auditory user interface of claim 10 further including thestep responsive to said aurally offering the user of the computercontrolled system said TTS user interface, for uninstalling saidinstalled TTS engine compatible with said routine when said user rejectssaid offered auditory user interface or when said TTS user declines anoffer to accept a license agreement or when said TTS user cancels saidinstallation of said application program during installation of saidapplication program.
 13. The method for providing said auditory userinterface of claim 11 further including the step responsive to saidaurally offering the user of the computer controlled system said TTSuser interface, for uninstalling said installed TTS engine compatiblewith said routine when said user rejects said offered auditory userinterface or when said TTS user declines an offer to accept a licenseagreement or when said TTS user cancels said installation of saidapplication program during installation of said application program. 14.The method for providing said auditory user interface of claim 8 whereinsaid step, responsive to the selection of said TTS interface forinstallation, for providing said TTS user interface during saidinstallation of said application program comprises the steps of:providing textual data representative of a sequence of displayable userinteractive dialog panels; and translating each of said sequence ofdialog panels into each of a corresponding sequence of interactiveauditory instruction and prompt panels.
 15. A computer program havingcode recorded on a computer readable medium for providing an auditoryuser interactive interface to an application program being installed inthe computer controlled system normally interfaced through a userinteractive display comprising: a routine in an application programbeing installed in said computer controlled system for providing a TTSuser interface to said program; TTS means for offering the user of thecomputer controlled system said TTS user interface during installationof said application program; and means, responsive to the selection ofsaid TTS interface for installation, for providing said TTS userinterface during said installation of said application program.
 16. Thecomputer program of claim 15 further including means for determiningwhether said computer controlled system includes a TTS engine compatiblewith said routine.
 17. The computer program of claim 16 furtherincluding means associated with said application program, responsive tosaid means for determining whether said computer controlled systemincludes a TTS engine, for installing a TTS engine compatible with saidroutine when the computer controlled system does not include said TTSengine.
 18. The computer program of claim 16 further including meansassociated with said application program, responsive to said determiningmeans determining that the computer controlled system does not include acompatible TTS engine, for seeking another compatible TTS system fromanother computer controlled system operatively associated with thecomputer controlled system; and means for installing said anothercompatible TTS system in said computer controlled system.
 19. Thecomputer program of claim 17 further including means, responsive to saidTTS means for offering the user of the computer controlled system saidauditory user interface, for uninstalling said installed TTS enginecompatible with said routine when said user rejects said offered TTSuser interface or when said TTS user declines an offer to accept alicense agreement or when said TTS user cancels said installation ofsaid application program during installation of said applicationprogram.
 20. The computer program of claim 15 wherein said means,responsive to the selection of said TTS interface for installation, forproviding said TTS user interface during said installation of saidapplication program comprises: means for providing textual datarepresentative of a sequence of displayable user interactive dialogpanels; and TTS means for translating each of said sequence of dialogpanels into each of a corresponding sequence of interactive auditoryinstruction and prompt panels.